Skip to content

Wire up InferenceController and RolloutExecutor and make the controller API async - #1843

Merged
fzyzcjy merged 1 commit into
mainfrom
tom/refactor-miles/op2-5
Sep 4, 2026
Merged

Wire up InferenceController and RolloutExecutor and make the controller API async#1843
fzyzcjy merged 1 commit into
mainfrom
tom/refactor-miles/op2-5

Conversation

@fzyzcjy

@fzyzcjy fzyzcjy commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Part of #1837

@fzyzcjy
fzyzcjy requested a review from yueming-yuan as a code owner July 28, 2026 12:55
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from b140598 to 0887196 Compare August 3, 2026 23:39
@fzyzcjy
fzyzcjy requested a review from Rockdu as a code owner August 3, 2026 23:39
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch 2 times, most recently from 8957d29 to 3b71f64 Compare August 4, 2026 00:39
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from 0887196 to 5f089d6 Compare August 4, 2026 06:56
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch from 3b71f64 to cd62224 Compare August 4, 2026 06:56
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from 5f089d6 to 1a71cad Compare August 10, 2026 06:06
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch from cd62224 to 44bfe59 Compare August 10, 2026 06:06
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from 1a71cad to cc79ac3 Compare August 10, 2026 16:07
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch from 44bfe59 to 23c4648 Compare August 10, 2026 16:07
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from cc79ac3 to 8dc50ea Compare August 10, 2026 16:34
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch 2 times, most recently from 788a07a to 2811f05 Compare August 12, 2026 02:48
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from 8dc50ea to fb28f27 Compare August 12, 2026 14:31
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch from 2811f05 to 20fa50d Compare August 12, 2026 14:31
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from fb28f27 to ee9ca13 Compare August 13, 2026 06:01
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch 2 times, most recently from aed6aba to 7848ec1 Compare August 14, 2026 04:46
@guapisolo

Copy link
Copy Markdown
Collaborator

Could we pull two small, dependency-light fixes forward and place them immediately after #1843, so this prefix can merge without waiting for the full refactor stack?

  1. Extract the minimal process-seam fix from Cut the eval fleet along the process seam #2614: keep the controller-side and executor-side EvalFleet state separate, and pass only data such as EvalFleetInfo / EvalFleetPin across the process boundary. This closes the serialized-copy/state-divergence bug introduced here. This should be described as a process-boundary correctness fix only: Delete the rollout health monitor ahead of its rewrite #1847 and Temporarily remove vanilla fault tolerance for the eval fleet #2497 intentionally remove the old eval-fleet FT path, so it must not claim that dedicated eval recovery is restored.
  2. Preserve replay compatibility for persisted audit JSONL with component="rollout_manager", either through a legacy alias or an explicit migration to rollout_executor. Let every miles process name itself as the component it is #2503 still accepts only rollout_executor, so older records remain unreadable or get dropped.

Ideally these are minimal follow-up commits/PRs directly on top of #1843, and we merge that smaller safe prefix before the remaining #1837 stack.

@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from 04b9e93 to c442051 Compare August 29, 2026 12:39
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch from 7848ec1 to 16cd06c Compare August 29, 2026 12:39
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from c442051 to cca973a Compare August 31, 2026 23:34
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch from 16cd06c to 42a5460 Compare August 31, 2026 23:34
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from cca973a to dafb45b Compare September 3, 2026 06:02
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch from 42a5460 to 0664348 Compare September 3, 2026 06:02
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from dafb45b to c9b2eba Compare September 4, 2026 00:53
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch from 0664348 to 56bd9d6 Compare September 4, 2026 00:54
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from c9b2eba to 90a12ad Compare September 4, 2026 03:28
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch from 56bd9d6 to 5cd7de2 Compare September 4, 2026 03:28
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-1 branch from 90a12ad to 149e93c Compare September 4, 2026 04:20
Base automatically changed from tom/refactor-miles/op2-1 to main September 4, 2026 04:20
…er API async

Squashed from:
- Run the inference controller in the driver and the executor as an actor
- Await the now-async rollout preparation
- Make the whole inference controller API async
- Make the async controller surface hold in the migration guide and its test
@fzyzcjy
fzyzcjy force-pushed the tom/refactor-miles/op2-5 branch from 5cd7de2 to c8c991e Compare September 4, 2026 04:20
@fzyzcjy
fzyzcjy merged commit 42736b9 into main Sep 4, 2026
2 of 10 checks passed
@fzyzcjy
fzyzcjy deleted the tom/refactor-miles/op2-5 branch September 4, 2026 04:20
@fzyzcjy
fzyzcjy restored the tom/refactor-miles/op2-5 branch September 4, 2026 07:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants